FIG. 1A shows components of an Oracle™ input and output (I/O) infrastructure. In the system shown in FIG. 1A, data may need to be copied over multiple times before it reaches its final destination during the performance of an I/O operation. Moreover, the operating system (O/S) kernel may need to be accessed in order that an I/O operation may be performed. In the system shown in FIG. 1A, to perform an I/O operation, an I/O application 101 communicates its I/O request to an ODM library 105 via an ODM API 103, the ODM library 105 in turn transmits the request to the O/S kernel 107 through an ODM interface or API 108. The O/S kernel 107 facilitates the access of the physical memory unit 111 via ODM driver 109 and may involve address translations. Physical memory accesses that involve the O/S kernel 107 exact a significant cost in time and processor usage that may be saved if such accesses may be avoided and/or reduced.
Computer data input and output (I/O) systems, such as that shown in FIG. 1A, facilitate the transfer of data to and from data storage units of computer systems. The performance of a computer systems data I/O system is directly related to the accessibility of the data that is located in the computer system's data storage units. Where data is readily available the speed of I/O operations may be significantly increased. Consequently, schemes for increasing the accessibility of stored data are of particular interest to computer system and software designers.
Buffer cache memory is volatile memory that may be accessed more quickly than data stored in disk. Data that is stored in cache memory is generally stored in data blocks. Input and output applications may initially search buffer cache memory for data before searching disk, e.g., slower, memory. If the data sought is found, then a more time consuming search of disk memory for the desired data may be avoided.
The performance of a database is related to the size of its buffer cache. The larger the buffer cache, the larger the number of data blocks that may be kept in main memory. Because of the increased number of data blocks that are available in main memory, the number of trips required to be taken to the slower disk subsystem or remote nodes may be reduced.
Virtual memory is a component of some conventional operating systems. The operating system moves data back and forth between non-volatile data storage units and volatile storage units of the computer to accommodate the memory demands of applications running on the computer.
The utilization of virtual memory increases the accessibility of stored data by allowing programmers to use a very large range of memory or storage addresses for stored data. Using such schemes frees a programmer from having to be concerned about the availability of memory space. The computing system maps the programmers virtual addresses to real hardware storage addresses (physical addresses) and stores them in a mapping table. Mapping translations that are stored in the mapping table may be accessed by I/O applications requesting the information represented by the entries listed therein. FIG. 1B shows a mapping table 150 such as may be employed to store the virtual address 153 to physical memory 151 address translations. As is shown in FIG. 1B, a virtual address 153 is mapped to an associated physical memory address 151.
Generally, operating systems support more virtual memory space than physical memory space. Computer systems built on X86 processors by Intel™ support more physical memory than may be addressed by the virtual space supported by the processor. It should be appreciated that in such systems, even though the hardware can support more physical memory, the amount of virtual address space available from the OS kernel is limited. Oracle's™ Extended Memory feature allows the extra physical memory that is supported to be exploited by mapping the buffer cache portion of the shared global area (SGA) into a small virtual window and by mapping and unmapping datablocks on demand. Using such a process a physical buffer may be mapped multiple times to different virtual addresses during the lifetime of a process.
Another way I/O operations may be improved is to allow input and output requests to be executed at the level of a network interface card (NIC). “User mode” I/O is a feature available in Oracle™ release that employs this concept. FIG. 2 illustrates the operation of the user mode I/O feature. In the system shown in FIG. 2, the I/O application 101 transmits an I/O request to an Oracle Disk Manager (ODM) library 103, and the ODM library 103 facilitates the access of the physical memory unit 111 via NIC 115. Since the physical memory access avoids involvement of the O/S kernel 107, a significant cost in time and processor usage may be saved. It should be appreciated that an Ethernet interface 117 (e.g., which may include a Direct Access File System and a Direct Access Provider Library DAPL) may be situated between the ODM library 103 and the NIC 115 to facilitate communication there between (117 includes an ODM driver).
As a part of the “user mode” I/O approach, input and output buffers may be registered with the NIC driver before they are used as described above. Once the buffers are registered with the NIC driver subsequent inputs and outputs from that buffer may be performed as a remote direct memory access (RDMA) operation at the level of the NIC without having to involve the OS kernel. This improves database performance due to a reduction in the OS kernel/code path and host central processing unit (CPU) usage.
However, taking full advantage of the data accessibility provided by both “extended memory” and “user mode” I/O presents serious challenges. This is because the advantages of these schemes may be lost due to the necessity of re-registering data buffers as a result of the limited amount of virtual address space that is available (because dynamic memory re-mapping is performed in X86 architectures). It should be appreciated that the number of buffers that reside in physical memory may be much larger than the number of buffers that can fit within a virtual memory window in the X86 architecture. Consequently, in order to accommodate all of the buffers that reside in physical memory the available virtual addresses may need to be dynamically used more than once.
The aforementioned user mode I/O process requires that the system explicitly map a virtual address to a corresponding physical buffer whenever the physical buffers contents are accessible, and that it subsequently unmap the virtual address from the physical buffer when the physical buffers contents become inaccessible. It should be appreciated that assigning a buffer to a new virtual address requires a re-registration in the NIC driver. The re-registration with the NIC driver involves a kernel context switch (e.g., a switch from the execution of one process on the CPU to another). It should be appreciated that an operating system kernel context switch involves an accessing of the operating system so that the benefits of user mode I/O may be eliminated and or significantly reduced in an environment that allows physical memory to virtual memory dynamic remapping, such as an X86 architecture.